DOCUMENT RESUME 



ED 482 278 



TM 035 308 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Mertler, Craig A. 

What .. .Another Survey??? Patterns of Response and Nonresponse 
from Teachers to Traditional and Web Surveys. 

2003-10-00 

25p.; Paper presented at the Annual Meeting of the Mid- 
Western Educational Research Association (Columbus, OH, 
October 15-18, 2003) . 

Reports - Research (143) -- Speeches/Meeting Papers (150) 

EDRS Price MF01/PC02 Plus Postage. 

*Mail Surveys; ^Research Methodology; ^Response Rates 
(Questionnaires) ; Responses; *World Wide Web 



ABSTRACT 

A study was conducted to compare the psychometric qualities 
of two identical forms of a survey administered in paper-and-pencil format 
and in Web format to 197 teachers randomly split into two paper and Web 
groups . ■ While the rates of response were quite discrepant, the respective 
measures of reliability were extremely similar for the two versions of the 
survey. A followup study was conducted with the nonrespondents in the Web 
group to investigate reasons behind their decision not to complete the 
survey. Several methodological issues are raised as a result of the followup. 
(Contains 3 tables and 16 references.) (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM035308 



What... Another Survey??? 
Patterns Of Response And Nonresponse 

From Teachers 

To Traditional And Web Surveys 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

c- l-ey 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



Craig A. Mertler, Ph.D. 
(mertler@bgnet.bgsu.edu ) 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
7 CENTER (ERIC) 

□/This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 

improve reproduction quality. 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Division of Educational Foundations & Inquiry 



School of Educational Leadership & Policy Studies 
College of Education & Human Development 



Bowling Green State University 
Bowling Green, OH 43403 



Paper presented at the annual meeting of the 
Mid-Western Educational Research Association, Columbus, OH (Oct. 15-18, 2003) 



Running head: Patterns Of Response And Nonresponse... 




2 



BEST COPY AVAILABLE 



Patterns Of Response And Nonresponse... 2 



Abstract 

This paper discusses the results of a study comparing the psychometric qualities of 
two forms of an identical survey: one administered in paper-and-pencil format and the 
other administered in Web format. Two groups of teachers were surveyed. One group 
received a paper-and-pencil version of the survey; the other group was directed to a Web- 
based version of the survey. While the rates of response were quite discrepant, the 
respective measures of reliability were extremely similar for the two versions of the 
survey. A follow-up survey was conducted with the nonrespondents in the Web group in 
order to investigate reasons behind their decision not to complete the survey. Several 
methodological issues are raised as a result of the follow-up. 
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What... Another Survey??? 

Patterns Of Response And Nonresponse From Teachers 
To Traditional And Web Surveys 

Background 

The Internet has had a substantial impact on the field of survey research (Shannon, 
Johnson, Searcy, & Lott, 2001). This is evidenced by the growing number of 
electronically administered surveys over the past several years. Web surveys— which 
Dillman (1998) has referred to as one type of “self-administered” survey— are an 
extremely promising method of data collection (Schillewaert, Langerak, & Duhamel, 
1998). 

Discussed in this background review of related literature are specific works related 
to (1) advantages and limitations of Web-based survey methodology, (2) methodological 
issues related to Web-based survey methodology, (3) psychometric qualities (i.e., 
response patterns) of Web-based versus traditionally administered surveys, and (4) 
patterns of nonresponse in Web-based surveys. 

Advantages and Limitations of Web-based Survey Methodology 

Advantages of Web surveys include a high rate of response, short time frame for the 
collection of responses, and time and cost savings. The Web certainly addresses the need 
for a less expensive and more expedient method of data collection (Solomon, 2001; 
Heflich & Rice, 1999; Schillewaert et al., 1998). Furthermore, several additional benefits 
of using the Web for data collection have been identified (Carbonaro & Bainbridge, 

2000; Mertler, 2001, 2002a, 2002b; Schillewaert et al., 1998; Shannon et al., 2001). 
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These benefits include a faster response, protection against the loss of data, easy transfer 
of data into a database for analysis, cost savings, convenience for the respondent, the 
possibility of wider geographic coverage, and a potentially better response rate— 
although this “advantage” is not uniformly agreed upon by the community of survey 
researchers (Matz, 1999). 

However, it is important to note, as with any method of data collection, there also 
exist disadvantages. These include the potentially nonrandom nature of the sample, 
unavailability of population lists, computer access to the survey, and various technology- 
related issues. Additional limitations include the inability to clearly define the population, 
lack of technological familiarity on the part of respondents or their willingness to use a 
computer to complete the survey, the potential for being able to identify respondents, and 
browser incompatibility problems (Solomon, 2001; Carbonaro & Bainbridge, 2000; 
Schillewaert et al., 1998; Shannon et al., 2001). 

One of the most substantial concerns about Web surveys is the potential nonrandom 
nature of the respondent group (Mertler, 2001, 2002a, 2002b; Solomon, 2001; Witte et 
al., 2000). However, the issue of nonrandomness is not unique to Web-based survey 
research and can be addressed through the maintenance of an accurate list of population 
members, when feasible. Survey research professionals have suggested that Web surveys 
be used primarily with specifically identifiable samples such as “in-house” employee 
groups (Shannon et al., 2001). Alternatively, Taylor (2000) has suggested that we 
remember that online data collection is not based on probability sampling, but rather on 
“volunteer” or “convenience” sampling. 
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Methodological Issues Related to Web-based Survey Methodology 

Beyond the strengths and weaknesses of this methodology, Carbonaro and 
Bainbridge (2000) have outlined several other issues with which researchers should be 
concerned. First, access to the survey must be as simple as possible for all respondents. 
The more complex the process of completing the survey, the lower the resultant response 
rate will undoubtedly be. Second, the process must be designed such that respondents of 
the Web survey are able to complete it with the same relative ease as if they had received 
a traditional paper version. Third, some sort of security system is required, in order to 
maintain the integrity of the data, but also to put at ease the mind of the respondent. 
Finally, completion of the survey must require only minimal computer skills— including 
the ability to use an Internet browser, enter a specific URL, use a mouse, and type on a 
word processor. Matz (1999) and Shannon et al. (2001) also point out that a Web survey 
must somehow be publicized. Some mechanism must be used to direct potential 
respondents to the actual URL containing the survey, either by providing a link to the 
URL in an email message or by providing the URL in a cover letter. 

Psychometric Qualities of Web-based Versus Traditionally Administered Surveys 

Although more and more studies comparing the effectiveness of electronic and mail 
surveys are being conducted, there exists somewhat of a void in the research literature on 
this topic (Underwood, Kim, & Matier, 2000). Often, the results of Web-based surveys 
differ when compared to written questionnaires and telephone surveys (Taylor, 2000), 
although Saphore (1999) found that there were no differences in the pattern of responses 
between a Web survey and an identical pencil-paper form of the same survey. 
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Furthermore, he concluded that there were no differences in the psychometric qualities of 
the two forms of the survey. 

In another comparison-type study, Matz (1999) compared the responses received 
from a Web survey with those received from a traditional paper-and-pencil survey. She 
found no significant differences between the demographic characteristics— such as age 
and gender— of the respondents completing the Web survey and those completing a 
mailed, written survey. Additionally, she found no significant differences between the 
content or pattern of responses for either group. The only significant finding— albeit, a 
substantial one— was that the overall response rate for the paper survey (43%) was 
significantly higher than that for the Web survey group (33%). Following his review of 
the literature, Solomon (2001) found this to be a consistent finding for studies which 
compared Web and mail surveys. A final result discussed by Matz (1999) was that the 
paper instrument seemed to provide more flexibility to the respondents. They could freely 
make comments about items they did not understand or felt were ambiguous. In spite of 
these last two findings, she concluded that Web surveys seemed to be a reasonable 
alternative— or, at least, equally suspect — to mail surveys. 

Another example of research comparing the psychometric characteristics of these 
two survey methodologies was conducted by Mertler and Earley (2003). The researchers 
concluded that Web-based and traditional survey methodologies result in similar sets of 
psychometric qualities based on comparable obtained values for internal consistency 
reliability. Although not identical, the patterns of responses— as identified by 
contributions to the overall scale— across the two modes of delivery were also fairly 
comparable. 
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In a final example, Idleman’s (2003) results contradicted those of Mertler and 
Earley (2003), with respect to the similarity of internal consistency values. Idleman 
(2003) observed that entire mail surveys, as well as subscales within those surveys, 
exhibited higher reliability coefficients that those resulting from a Web-based version of 
the same survey, although the coefficients were well within the acceptable range for both 
modes of delivery. 

Patterns of Nonresponse in Web-based Surveys 

A viable and important source of error in surveys is nonresponse (Montez, 2003). 
The reason for this is that the characteristics of nonrespondents may somehow differ from 
those of respondents, potentially limiting the external validity of the survey’s results 
(Montez, 2003). 

Solomon (2001) notes that some potential respondents have difficulty with the 
technology and give up early in the process of completing a Web-based survey, or when 
encountering complex questions. Others may often be reluctant to give out personal 
information, such as an email address. 

In her study of nonresponse, Montez (2003) received 55 (25%) follow-up responses 
from 218 original nonrespondents. In analyzing the reasons given for nonresponse, five 
categories emerged. The five categories were: 

• simple, but polite, refusal to participate; 

• not enough time to participate; 

• change in professional position (therefore, the topic of the survey was no longer 
appropriate); 
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• desire to only respond to surveys that are prepared as part of the business of 
national organizations; and 

• belief that the survey was poorly designed or did not truly capture the desired 
topic. 

Summary 

Because of the technological skills required to develop and implement Web-based 
surveys, the driving force behind their use has not been survey professionals, but rather 
technology specialists (or at least someone with a background in technology) (Dillman & 
Bowker, 2001; Shannon et al., 2001). In order to harness the potential for using the 
Internet for the collection of valid and reliable data, those most knowledgeable about 
survey research methodology— specifically with respect to causes and consequences of 
survey error (Dillman & Bowker, 2001)— must take an active role in its development as a 
viable methodology for data collection. Specifically, research is needed in order to 
compare responses from Web surveys and those conducted in more traditional manners 
(Dillman & Bowker, 2001). 

Purpose of the Study 

The primary purpose of this study was to add to the knowledge base regarding the 
use of Web-based surveys as a viable means of collecting data for educational research. 
Specifically, the researchers investigated and compared the relative effectiveness, 
psychometric qualities, and response patterns of two versions of the same survey: one 
delivered as a Web-based survey and the other as a paper survey. A secondary purpose 
was to assess the patterns of nonresponse, particularly with respect to the Web version of 
the survey. 
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The research questions addressed in the study were: 

Research Question 1 : What are the psychometric qualities (i.e., response patterns) for 
survey data resulting^ from traditional and Web-based delivery modes? How do the 
two sets of qualities compare? 

Research Question 2 : What patterns of survey nonresponse result from the administration 
of the same survey? 

Methods 

Participants 

During the fall of 2002, the researcher surveyed inservice teachers with respect to 
their assessment literacy. The group of inservice teachers consisted of 197 teachers, 
representing nearly every district in a three-county area surrounding the researcher’s 
institution. The schools were selected based on convenience due to their geographic 
location. All grade levels and content were represented in the final sample. The teachers 
were randomly split into two groups: one group received a paper-and-pencil version of 
the survey; the other group was directed to a Web-based version of the survey. 
Instrumentation 

Both groups of teachers were originally surveyed using an instrument titled the 
Classroom Assessment Literacy Inventory, or CALI, which was adapted from a similar 
instrument called the Teacher Assessment Literacy Questionnaire (Plake, 1993; Plake, 
Impara, & Fager, 1993). This inventory is based on the Standards for Teacher 
Competence in the Educational Assessment of Students (AFT, NCME, & NEA, 1990). 
The CALI consisted of the same 35 content-based items (five per standard) with a limited 
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amount of rewording (e.g., changing some names of fictitious teachers, changing word 
choice to improve clarity, etc.), as well as 7 demographic items. 

The original instrument has been shown to have reasonable reliability with inservice 
teachers, r KR _ 20 = .54 (Plake, Impara, & Fager, 1993). Additionally, the original 
instrument was subjected to a thorough content validation, including reviews by members 
of the National Council on Measurement in Education and a pilot study with and 
feedback from practicing teachers and administrators. 

Procedures 

The paper-and-pencil group received the cover letter and survey directly through 
U.S. Mail and was supplied with a postage-paid return envelope. The Web-based group 
received an email message containing the cover “letter” and a link consisting of the URL 
to the survey. Two weeks after the initial mailing of the paper version and posting of the 
Web-based version, teachers were sent a reminder about completing the instrument. 

Due to the low rate of response to the Web version of the survey, a follow-up 
“survey” was conducted via email for the group of nonrespondents. This email survey 
consisted of the following: 



I received a very poor response to my recent Web survey titled the Classroom 
Assessment Literacy Inventory. I am attempting find out why the response was so low. 

If you did not complete the survey, please take 30 seconds to reply to this email 
message and indicate which of the reasons listed below was the main reason you did 
not complete the survey. You can do so by simply typing an "X" next to you response. 
If you select "Other," please provide a brief explanation. 

( ) The topic didn't interest me. 

( ) I couldn't access the survey due to limitations of technology. 

( ) I was afraid of the security/conf identiality of my responses. 

( ) I couldn't access the survey due to my lack of technological expertise. 

( ) I simply didn't want to take the time to respond. 

( ) The survey was too lengthy. 

( ) Other (please explain) 

Thank you very much! ! ! 

Craig A. Mertler, Ph.D. 
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Analyses 

For the two modes of survey delivery, statistical analyses included the computation 
of internal consistency (i.e., coefficient alpha) reliability coefficients and individual item 
analyses. All statistical analyses were conducted using SPSS (v. 11). Content analyses 
were also conducted for the teacher-supplied reasons for nonresponse in an attempt to 
classify them into thematic categories. 

Results 

The results that follow are presented by each individual research question. 

Research Question 1 : What are the psychometric qualities (i.e., response patterns) 

for survey data resulting from traditional and Web-based delivery modes? 

How do the two sets of qualities compare? 

Although the data resulting from the administration of the two surveys were 
analyzed for the entire group, the analyses were primarily based on “formal” (i.e., 
statistical) techniques followed by “informal” (i.e., nonstatistical) comparisons of the 
results by the two subgroups— namely, the data resulting from the traditional paper- and- 
pencil method of delivery (N = 142) and those resulting from the Web form of the survey 
(N = 55). The return rate for the total survey was equal to 17%; the return rate for the 
paper-and-pencil administration was equal to 21%, while that for the Web administration 
was equal to 11%. 

Analysis of the overall scale comprised of 35 items for the entire group (N = 197) 
revealed an internal consistency measure (i.e., Cronbach’s index of internal consistency; 
also known as the alpha coefficient, or a) equal to .57, indicating a moderate degree of 
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reliability. The internal consistency measure for the “paper” sample (a = .56) was nearly 
identical to that for the “Web” sample (a = .59). 

The resulting data were also analyzed at the level of the individual items. 
Specifically, the internal consistency of the overall scale with each individual item 
removed was determined. This is typically done in order to assess the contributions of 
individual items to the total scale. The resulting analysis reports an alpha (a) coefficient 
for the total scale minus the particular item. A “new” a coefficient (i.e., for the 34-item 
scale) that is lower than that for the 35-item scale indicates that the particular item did in 
fact contribute positively to the total scale; in other words, when the item was removed, 
the overall reliability decreased. In contrast, a “new” a coefficient that is higher than that 
for the original scale indicates that the particular item did not contribute to the overall 
scale; in other words, when the item was removed, the overall reliability improved. 

Of the 35 items appearing on the survey administered in a traditional manner (i.e., 
paper-and-pencil), analyses of 6 items revealed improved internal consistency values 
when they were individually removed from the scale. The results of the individual item 
analyses for the survey administered via the Web were somewhat similar. Of the 35 
items, 10 showed improved internal consistencies when removed from the scale. A 
summary of the analyses of these individual items and the amounts of improvement in 
overall scale reliabilities are presented in Table 1 and Table 2, respectively. Furthermore, 
it is important to note that, upon informal comparison of the list of 6 items identified 
from the paper-and-pencil method and the 10 items identified from the Web delivery 
method, 4 items were common to both lists. 
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Insert Table 1 here 



Insert Table 2 here 



Research Question 2 : What patterns of survey nonresponse result from the 

administration of the same survey? 

A potentially more interesting set of findings resulted from the originally 
unintended follow-up survey conducted with those teachers in the Web group who chose 
not to complete the survey of primary interest in this study. These teachers (N = 457) 
were emailed and asked to indicate from seven options the main reason they did not 
respond to the survey. Ninety-four teachers, representing 21% of the nonrespondent 
group responded to the follow-up (interestingly, this was nearly twice the number that 
responded to the original survey). The frequencies of response to the seven options are 
presented in Table 3. The teachers were instructed to explain their response if they 
marked option #7 (“Other”). However, many teachers provided additional comments, 
regardless of the option they selected. 



Insert Table 3 here 



The most common reason for nonresponse to the original survey was “I simply 
didn’t want to take the time to respond,” with over one-third (37%) of nonrespondents 
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indicating that this was their reason. Examples of open-ended comments that fit into this 
category included the following: 



I didn’t have the time, no matter how short or long. 

I simply do not have the time to do this at this time. 1 could maybe do it 
this summer. 

My day is packed FULL with virtually no time to spare. It’s not that I 
didn’t WANT to take the time. . . other necessary tasks had absolute 
priority on my time. . . and if you have ever taught a core high school 
subject for any length of time, you’d know that. 

I started doing the survey and it was taking longer that I cared to spend 
doing the survey. 1 stopped doing the survey after about 3-4 questions. 



A considerable number of teachers (16%) also indicated that they did not respond 
due to technological limitations (e.g., couldn’t access the survey, lack of technological 
expertise, etc.). Examples of open-ended comments that were categorized here included 
the following: 



I am sorry I didn’t reply, but I am not able to open the survey. 

I couldn’t reply. You had a link to click on and nothing happened when 1 
clicked on it. Also, I couldn ’t put an X in the spot you asked me to on this 
email. 

I didn’t fill out the survey because the hyperlink did not connect me. 1 
didn’t want to take the time to fill in the address. 

Can’t access the survey or type an “X” in the appropriate answer below. 

Our set-up at [our school] is VERY limited. We cannot open the document 
- 1 tried, but the computer would not allow this to happen. 

I did not respond to you survey because 1 could not get into your site. 

I was unable to respond to your survey because of poor school equipment. 




15 



Patterns Of Response And Nonresponse... 15 



Began answering the questions. When my planning period was over, I 
stopped answering. When I came back the next day, none of my answers 
had been saved. I simply do not have the time to spend two planning 
periods answering a survey. 

...I am the tech rep for my building. . . three other teachers in the building 
sought my help - two of them had the same trouble 1 did - it disappeared 
without sending upon completion. The third teacher responded to 
approximately 1/2 and attempted to save her responses so she could 
complete it at another date. When she went back to it, all of her answers 
were gone. 



Fourteen teachers (15%) indicated that the survey was simply too long to respond 
to. Many of those who checked this option also checked the option stating that they did 
not want to take the time to respond. 

Finally, twenty-four teachers (26%) checked “Other.” These teachers offered a 
variety of explanations for their selection. These explanations included the following as 
examples: 

I chose not to answer your survey due to confidentiality reasons. There 
was no way to respond without being identified. 

Everyone gets tons of junkmail. 1 delete everything that is not from 
someone I know. 

/ deleted the message accidentally and don’t know where the survey is. 

As a kindergarten teacher, 1 do not do much to assess literacy. 

This was not in my area of expertise. 



Discussion 

Research has begun to demonstrate that utilizing a Web-based approach is a viable 
means of gathering survey data. However, further research must be conducted on this 
mode of delivery in order to pass judgment on its relative merits. There are several 
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advantages to electronic surveys, in general. These include such things as cost and 
timesavings, as well as the ease of transfer of responses into a database. Are these 
advantages over paper-and-pencil surveys great enough to “counteract” the problems 
associated with nonrandom samples? Are potential respondents comfortable enough with 
technology to respond to surveys online? Or do they remain apprehensive about 
potentially realistic concerns such as anonymity, confidentiality, and security of their 
provided information? If the answer to the latter question is “yes”— which certainly 
seemed to be the case in the present study— do we as researchers end up with a 
“differential” type of random sample resulting from respondent self-selection? This, of 
course, is a concern for any type of survey research, but can we be sure that we are not 
ending up with a sample whose characteristics are different from those realized through 
paper-and-pencil forms of surveys? To address these and related issues, further research 
is most certainly called for. 

A few interesting methodological issues related to the Web-based mode of survey 
delivery were raised as a result of this study. First, the researcher, working in consultation 
with an information technology professional at the institution, checked the viability of the 
Web-based survey on a variety of Web browsers (e.g., Internet Explorer, Netscape 
Navigator, and others), as well as various versions of those browsers. Although these 
various combinations of browsers and versions were verified, some teachers still were 
unable to access the survey, as evidenced by their comments in response to the follow-up 
survey. This may have been due— at least in part— to older hardware and/or software in 
the schools. 



ERIC 



17 



Patterns Of Response And Nonresponse... 17 

Second, it is important to note that many of the teachers surveyed in this study 
worked in rural school settings. Many of those districts contract with private Internet 
service providers (ISPs) to provide Internet access and email services. Another possible 
cause for teachers’ inabilities to access the survey may have been caused by the various 
settings provided by the ISPs who supply service to these districts. Following the initial 
email message (i.e., cover letter) for the primary survey, several teachers corresponded 
with the researcher to inform him that their district did not support hyperlinks embedded 
in email messages. In some instances, the URL appeared in the message, and when 
teachers clicked on it, nothing happened (of course, this is easily remedied by copying 
and pasting the link into the URL bar of a browser window). In other situations, the URL 
did not even appear in the body of the email message. Still other teachers informed the 
researcher that their district did not permit delivery of email messages from unknown 
sources or access to unknown URLs. It is possible then that for some teachers access to 
the survey was literally impossible (and, therefore, not a conscience decision made on the 
part of individual teachers), or that some of them never even received the email cover 
letter. Obviously, this would have had a substantial impact on the ultimate rate of 
response to the survey. 

Third, and somewhat related to the second issue above, involves the issue of 
technology literacy. Although we have a tendency to believe that everyone— especially in 
the educational community— in this day and age is technologically literate (e.g., knows 
all about browsers, how to maneuver around the World Wide Web, how to alter email 
settings, etc.), the fact remains that many individuals, especially teachers, are simply not 
literate in this sense. Many teachers, when encountered with a hyperlink that did not 
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“work,” were apparently unaware that the URL could be copied and pasted into a 
browser window; they simply gave up and did not try further to access the survey. Also 
troublesome to the researcher was the fact that some teachers began to respond to the 
survey, logged off of their computers, logged back onto them the next day, and expected 
to gain access to their partially completed survey, ready to finish and submit. Most— if 
not all— Web pages do not work this way. 

As further evidence of this phenomenon of technology //-literacy, witness the 
following brief email conversation held with one of the nonresponding teachers as a 
result of the follow-up survey, who clearly did not know what a Web browser even was: 

Teacher: 1 have been unable to access your literacy survey. I have tried 

several times, but it doesn’t seem to want to come up on my 
computer. 

Researcher: What type of Web browser are you using? Do you know the 

version number? 

Teacher: No clue! I’m on [private computer service provider J. Does that 

help you? 

Generally speaking, further research on the feasibility of the Web-based delivery of 
surveys is most certainly warranted. In addition, as a result of this study, other vitally 
important methodological issues have had some light shed upon them. The issue of equal 
access to Web-based surveys— and to electronic surveys, in general— as well as the issue 
of the technological capabilities of potential respondents truly calls into question the 
extent to which educators, especially in K-12 settings, should be surveyed via electronic 
means. At this point in time, if the educational research community continues to do so, 
we can only assume that we are, in all probability, obtaining anything but a representative 
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Table 1 

Identified Items and Recalculated Reliabilities for Item Analysis of Data Resulting from 
Paper-and-pencil Survey 



Survey Item Number Alpha Coefficient 

(If Item Deleted) a 


Amount of Change 
In Reliability 


Item 5 b 

i 


.590 


1 

+.033 ; 


Item 10 


.564 


+.040 


Item 13 b 


.561 


" ' i 

+.044 ; 

1 

j 


Item 21 


.559 


+.046 


Item 22 b 


.561 


1 

+.050 


Item 31 b 


.578 


+.071 



a Alpha coefficient of the total scale was equal to .557 (rounded to .56). 
b These items were also identified following the item analysis of data resulting from the 
Web-administered survey. 
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Table 2 

Identified Items and Recalculated Reliabilities for Item Analysis of Data Resulting from 



Web Survey 



Survey Item Number 


Alpha Coefficient 
(If Item Deleted) a 


Amount of Change 
In Reliability 


Item 2 


.598 


+.005 


Item 4 


.598 


+.005 


Item 5 b 


.610 


+.022 j 

1 


Item 7 


.594 


+.023 


Item 1 3 b 


.604 


+.034 


Item 14 


.6,13 


+.020 


Item 16 


.597 


+.024 


Item 22 b 


.597 


+.004 


Item 26 


.609 


+.016 


Item 3 1 b 

i 


.604 


+.011 



a Alpha coefficient of the total scale was equal to .593 (rounded to .89). 
b These items were also identified following the item analysis of data resulting from the 
paper-and-pencil-administered survey. 
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Table 3 

Frequencies and Percentages of Response for Follow-up Survey of Nonrespondents 



Reason 


Frequency of 
Response 


Percentage of 
Response 


The topic didn’t interest me. 


1 


1% 


1 couldn’t access the survey due to 
limitations of technology. 


15 


16% 


I was afraid of the 

security/ confidentiality of my responses. 


4 


4% 


I couldn’t access the survey due to my 
lack of technological expertise. 


1 


1% 


I simply didn ’t want to take the time to 
respond. 


35 


37% 


The survey was too lengthy. 


14 


15% 


Other 


24 


26% 
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